A homotopy training algorithm for fully connected neural networks

Universality of Fully-Connected Recurrent Neural Networks

It is shown from the universality of multi-layer neural networks that any discretetime or continuous-time dynamical system can be approximated by discrete-time or continuous-time recurrent neural networks, respectively.

Autonomous learning algorithm for fully connected recurrent networks

In this paper fully connected RTRL neural networks are studied. In order to learn dynamical behaviours of linear-processes or to predict time series, an autonomous learning algorithm has been developed. The originality of this method consists of the gradient based adaptation of the learning rate and time parameter of the neurons using a small perturbations method. Starting from zero initial con...

A Learning Algorithm for Continually Running Fully Recurrent Neural Networks

The exact form of a gradient-following learning algorithm for completely recurrent networks running in continually sampled time is derived and used as the basis for practical algorithms for temporal supervised learning tasks. These algorithms have: (1) the advantage that they do not require a precisely deened training interval, operating while the network runs; and (2) the disadvantage that the...

Conjugate gradient algorithm for training neural networks

On global asymptotic stability of fully connected recurrent neural networks

Conditions for Global Asymptotic Stability (GAS) of a nonlinear relaxation process realized by a Recurrent Neural Network (RNN) are provided. Existence. convergence, and robustness of such a process are analyzed. This is undertaken based upon the Contraction Mapping Theorein (CMT) and the corresponding Fixed Point Iteration (FPI). Upper bounds for such a process are shown to be the conditions o...